Ultra-Wideband Radar-Based Activity Recognition Using Deep Learning

With recent advances in the field of sensing, it has become possible to build better assistive technologies. This enables the strengthening of eldercare with regard to daily routines and the provision of personalised care to users. For instance, it is possible to detect a person’s behaviour based on wearable or ambient sensors; however, it is difficult for users to wear devices 24/7, as they would have to be recharged regularly because of their energy consumption. Similarly, although cameras have been widely used as ambient sensors, they carry the risk of breaching users’ privacy. This paper presents a novel sensing approach based on deep learning for human activity recognition using a non-wearable ultra-wideband (UWB) radar sensor. UWB sensors protect privacy better than RGB cameras because they do not collect visual data. In this study, UWB sensors were mounted on a mobile robot to monitor and observe subjects from a specific distance (namely, 1.5–2.0 m). Initially, data were collected in a lab environment for five different human activities. Subsequently, the data were used to train a model using the state-of-the-art deep learning approach, namely long short-term memory (LSTM). Conventional training approaches were also tested to validate the superiority of LSTM. As a UWB sensor collects many data points in a single frame, enhanced discriminant analysis was used to reduce the dimensions of the features through application of principal component analysis to the raw dataset, followed by linear discriminant analysis. The enhanced discriminant features were fed into the LSTMs. Finally, the trained model was tested using new inputs. The proposed LSTM-based activity recognition approach performed better than conventional approaches, with an accuracy of 99.6%. We applied 5-fold cross-validation to test our approach. We also validated our approach on publically available dataset. The proposed method can be applied in many prominent fields, including human–robot interaction for various practical applications, such as mobile robots for eldercare.publishedVersio

User-adaptive models for activity and emotion recognition using deep transfer learning and data augmentation

Kan bare brukes i forskningssammenheng, ikke kommersielt. Les mer her: https://www.springernature.com/gp/open-research/policies/accepted-manuscript-termsBuilding predictive models for human-interactive systems is a challenging task. Every individual has unique characteristics and behaviors. A generic human–machine system will not perform equally well for each user given the between-user differences. Alternatively, a system built specifically for each particular user will perform closer to the optimum. However, such a system would require more training data for every specific user, thus hindering its applicability for real-world scenarios. Collecting training data can be time consuming and expensive. For example, in clinical applications it can take weeks or months until enough data is collected to start training machine learning models. End users expect to start receiving quality feedback from a given system as soon as possible without having to rely on time consuming calibration and training procedures. In this work, we build and test user-adaptive models (UAM) which are predictive models that adapt to each users’ characteristics and behaviors with reduced training data. Our UAM are trained using deep transfer learning and data augmentation and were tested on two public datasets. The first one is an activity recognition dataset from accelerometer data. The second one is an emotion recognition dataset from speech recordings. Our results show that the UAM have a significant increase in recognition performance with reduced training data with respect to a general model. Furthermore, we show that individual characteristics such as gender can influence the models’ performance.acceptedVersio

Monitoring In-Home Emergency Situation and Preserve Privacy using Multi-modal Sensing and Deep Learning

Videos and images are commonly used in home monitoring systems. However, detecting emergencies in-home while preserving privacy is a challenging task concerning Human Activity Recognition (HAR). In recent years, HAR combined with deep learning has drawn much attention from the general public. Besides that, relying entirely on a single sensor modal-ity is not promising. In this paper, depth images and radar presence data were used to investigate if such sensor data can tackle the challenge of a system's ability to detect abnormal and normal situations while preserving privacy. The recurrence plots and wavelet transformations were used to make a two-dimensional representation of the presence radar data. Moreover, we fused data from both sensors using data-level, feature-level, and decision-level fusions. The decision-level fusion showed its superiority over the other two techniques. For the decision-level fusion, a combination of the depth images and presence data recurrence plots trained first on convolutional neural networks (CNN). The output was fed into support vector machines, which yielded the best accuracy of 99.98%.acceptedVersio

Enabling Participants to Play Rhythmic Solos Within a Group via Auctions

The paper presents the interactive music system SoloJam, which allows a group of participants with little or no musical training to effectively play together in a ``band-like'' setting. It allows the participants to take turns playing solos made up of rhythmic pattern sequences. We specify the issue at hand for allowing such participation as being the requirement of decentralised coherent circulation of playing solos. This is to be realised by some form of intelligence within the devices used for participation. Here we take inspiration from the Economic Sciences, and propose this intelligence to take the form of making devices possessing the capability of evaluating their utility of playing the next solo, the capability of holding auctions, and of bidding within them. We show that holding auctions and bidding within them enables decentralisation of co-ordinating solo circulation, and a properly designed utility function enables coherence in the musical output. The approach helps achieve decentralised coherent circulation with artificial agents simulating human participants. The effectiveness of the approach is further supported when human users participate. As a result, the approach is shown to be effective at enabling participants with little or no musical training to play together in SoloJam

Complexity and variability analyses of motor activity distinguish mood states in Bipolar Disorder

Changes in motor activity are core symptoms of mood episodes in bipolar disorder. The manic state is characterized by increased variance, augmented complexity and irregular circadian rhythmicity when compared to healthy controls. No previous studies have compared mania to euthymia intra-individually in motor activity. The aim of this study was to characterize differences in motor activity when comparing manic patients to their euthymic selves. Motor activity was collected from 16 bipolar inpatients in mania and remission. 24-h recordings and 2-h time series in the morning and evening were analyzed for mean activity, variability and complexity. Lastly, the recordings were analyzed with the similarity graph algorithm and graph theory concepts such as edges, bridges, connected components and cliques. The similarity graph measures fluctuations in activity reasonably comparable to both variability and complexity measures. However, direct comparisons are difficult as most graph measures reveal variability in constricted time windows. Compared to sample entropy, the similarity graph is less sensitive to outliers. The little-understood estimate Bridges is possibly revealing underlying dynamics in the time series. When compared to euthymia, over the duration of approximately one circadian cycle, the manic state presented reduced variability, displayed by decreased standard deviation (p = 0.013) and augmented complexity shown by increased sample entropy (p = 0.025). During mania there were also fewer edges (p = 0.039) and more bridges (p = 0.026). Similar significant changes in variability and complexity were observed in the 2-h morning and evening sequences, mainly in the estimates of the similarity graph algorithm. Finally, augmented complexity was present in morning samples during mania, displayed by increased sample entropy (p = 0.015). In conclusion, the motor activity of mania is characterized by altered complexity and variability when compared within-subject to euthymia.publishedVersio

Behavioural Plasticity Can Help Evolving Agents in Dynamic Environments but at the Cost of Volatility

Neural networks have been widely used in agent learning architectures; however, learnings for one task might nullify learnings for another. Behavioural plasticity enables humans and animals alike to respond to environmental changes without degrading learned knowledge; this can be achieved by regulating behaviour with neuromodulation—a biological process found in the brain. We demonstrate that by modulating activity-propagating signals, neurally trained agents evolving to solve tasks in dynamic environments that are prone to change can expect a significantly higher fitness than non-modulatory agents and also achieve their goals more often. Further, we show that while behavioural plasticity can help agents to achieve goals in these variable environments, this ability to overcome environmental changes with greater success comes at the cost of highly volatile evolution

Direct brain recordings reveal continuous encoding of structure in random stimuli

The brain excels at processing sensory input, even in rich or chaotic environments. Mounting evidence attributes this to the creation of sophisticated internal models of the environment that draw on statistical structures in the unfolding sensory input. Understanding how and where this modeling takes place is a core question in statistical learning and predictive processing. In this context, we address the role of transitional probabilities as an implicit structure supporting the encoding of a random auditory stream. Leveraging information-theoretical principles and the high spatiotemporal resolution of intracranial electroencephalography, we analyzed the trial-by-trial high-frequency activity representation of transitional probabilities. This unique approach enabled us to demonstrate how the brain continuously encodes structure in random stimuli and revealed the involvement of a network outside of the auditory system, including hippocampal, frontal, and temporal regions. Linking the frame-works of statistical learning and predictive processing, our work illuminates an implicit process that can be crucial for the swift detection of patterns and unexpected events in the environment.Fil: Fuhrer, Julian. University of Oslo; NoruegaFil: Kyrre, Glette. University of Oslo; NoruegaFil: Ivanovic, Jugoslav. University of Oslo; NoruegaFil: Gunnar Larsson, Pål. University of Oslo; NoruegaFil: Bekinschtein, Tristán Andrés. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. University of Cambridge; Reino UnidoFil: Kochen, Sara Silvia. Universidad Nacional Arturo Jauretche. Unidad Ejecutora de Estudios en Neurociencias y Sistemas Complejos. Provincia de Buenos Aires. Ministerio de Salud. Hospital Alta Complejidad en Red El Cruce Dr. Néstor Carlos Kirchner Samic. Unidad Ejecutora de Estudios en Neurociencias y Sistemas Complejos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata. Unidad Ejecutora de Estudios en Neurociencias y Sistemas Complejos; ArgentinaFil: Knight, Robert T.. University of California at Berkeley; Estados UnidosFil: Tørresen, Jim. University of Oslo; NoruegaFil: Solbakk, Anne Kristin. University of Oslo; Noruega. Helgeland Hospital; NoruegaFil: Endestad, Tor. University of Oslo; Noruega. Helgeland Hospital; NoruegaFil: Blenkmann, Alejandro Omar. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. University of Oslo; Norueg

Towards Adaptive Technology in Routine Mental Healthcare

This paper summarizes the information technology-related research findings after 5 years with the INTROducing Mental health through Adaptive Technology project. The aim was to improve mental healthcare by introducing new technologies for adaptive interventions in mental healthcare through interdisciplinary research and development. We focus on the challenges related to internet-delivered psychological treatments, emphasising artificial intelligence, human-computer interaction, and software engineering. We present the main research findings, the developed artefacts, and lessons learned from the project before outlining directions for future research. The main findings from this project are encapsulated in a reference architecture that is used for establishing an infrastructure for adaptive internet-delivered psychological treatment systems in clinical contexts. The infrastructure is developed by introducing an interdisciplinary design and development process inspired by domain-driven design, user-centred design, and the person based approach for intervention design. The process aligns the software development with the intervention design and illustrates their mutual dependencies. Finally, we present software artefacts produced within the project and discuss how they are related to the proposed reference architecture. Our results indicate that the proposed development process, the reference architecture and the produced software can be practical means of designing adaptive mental health care treatments in correspondence with the patients’ needs and preferences. In summary, we have created the initial version of an information technology infrastructure to support the development and deployment of Internet-delivered mental health interventions with inherent support for data sharing, data analysis, reusability of treatment content, and adaptation of intervention based on user needs and preferences.publishedVersio

On Assessing Trustworthy AI in Healthcare. Machine Learning as a Supportive Tool to Recognize Cardiac Arrest in Emergency Calls

Artificial Intelligence (AI) has the potential to greatly improve the delivery of healthcare and other services that advance population health and wellbeing. However, the use of AI in healthcare also brings potential risks that may cause unintended harm. To guide future developments in AI, the High-Level Expert Group on AI set up by the European Commission (EC), recently published ethics guidelines for what it terms “trustworthy” AI. These guidelines are aimed at a variety of stakeholders, especially guiding practitioners toward more ethical and more robust applications of AI. In line with efforts of the EC, AI ethics scholarship focuses increasingly on converting abstract principles into actionable recommendations. However, the interpretation, relevance, and implementation of trustworthy AI depend on the domain and the context in which the AI system is used. The main contribution of this paper is to demonstrate how to use the general AI HLEG trustworthy AI guidelines in practice in the healthcare domain. To this end, we present a best practice of assessing the use of machine learning as a supportive tool to recognize cardiac arrest in emergency calls. The AI system under assessment is currently in use in the city of Copenhagen in Denmark. The assessment is accomplished by an independent team composed of philosophers, policy makers, social scientists, technical, legal, and medical experts. By leveraging an interdisciplinary team, we aim to expose the complex trade-offs and the necessity for such thorough human review when tackling socio-technical applications of AI in healthcare. For the assessment, we use a process to assess trustworthy AI, called 1Z-Inspection® to identify specific challenges and potential ethical trade-offs when we consider AI in practice.</jats:p

A Review of Future and Ethical Perspectives of Robotics and AI

In recent years, there has been increased attention on the possible impact of future robotics and AI systems. Prominent thinkers have publicly warned about the risk of a dystopian future when the complexity of these systems progresses further. These warnings stand in contrast to the current state-of-the-art of the robotics and AI technology. This article reviews work considering both the future potential of robotics and AI systems, and ethical considerations that need to be taken in order to avoid a dystopian future. References to recent initiatives to outline ethical guidelines for both the design of systems and how they should operate are included